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What's changed since Apollo? 
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MER, Sojourner, MSL. 
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Ss nen-coro teaming for space exploration 


is before crew 

¢ Prepare for subsequent human mission 

¢ Scouting, prospecting, etc. = 

¢ Site preparation, equipment deployment, yas 
infrastructure setup, etc. _— 


is after cr 
¢ Perform work following human mission 
¢ Follow-up work 

¢ Close-out tasks, maintenance, etc. 


Robots and crew 

° Extend and erinanes human reach 

¢ Parallel activities and remote operations 
¢ Inspection, mobile camera, etc. 
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NASA Robotic Recon Experiment 


Objectives 


¢ Test coordinated human-robot 
field exploration 


¢ Robot scouts ahead of crew 


¢ Fold lessons learned into lunar 
surface science ops concepts 


Results 
¢ Identified requirements (instruments, 
comm, nav, etc.) for robotic recon 
¢ Assessed impact of robotic recon on 
traverse planning & crew productivity 
¢ Learned how to improve human 
productivity & science return 


M. Bualat et al. (2011). Robotic recon for 


human exploration: method, assessment, 
and lessons learned. GSA Special Paper. 
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Field experiment (2009) 


Mission Planning » Robot Mission 


Mar 1 — June 1 June 14 — June 26 July 1 — Aug 15 

¢ Satellite images ¢ K10 at Black Point ¢ Recon images 

* Geologic map ¢ Ground control at ¢ Terrain models 
<>. NASA Ames 
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Aug 29 - Sep 3 
¢ LER at BPLF 


¢ Science back- 
room at BPLF 
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Black Point Lava Flow 
¢ 65 km N of Flagstaff, AZ 


¢ Analog of the “Straight Wall” 
(Mare Nubrium / Rupes Recta) 


The “Straight Wall” 


15 km 


Black Point Lava Flow 
sn teaming for space exploration 


Google 


Robot mission (June 2009) 
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Google 


8.5 GB data collected (52 hrs of robotic recon operations) 
39 LIDAR scans, 75 GigaPan, and 95 terrain images 
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100% scale 


Image USDA Farm Service Agency 


2009 Tele Atlas 


Digital Globe QuickBird (60 cm/pixel) 
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3D scanning LIDAR (250 m range, 3 mm depth resolution) 
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Space Exploration Vehicle (SEV) 
¢ Prototype pressurized crew 
vehicle for lunar operations 
¢ Two “suit ports” for rapid (15 min) 
egress and ingress 
¢ 20 km/hr max, active suspension 
¢ 3.5 x 5 m (wheelbase x length) 


Crew A 
¢ Mike Gernhardt & Brent Garry 


¢ W1 (pre-recon) + N2 (post-recon) 
traverses 


Crew B 
¢ Andy Thomas & Jake Bleacher 


¢ N1 (pre-recon) + W2 (post-recon) 
traverses 
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Field experiment results 


“West” region 
¢ Pre-recon traverse plan was 
designed like Apollo 
= Assume single visit to each site 


= Rapid area coverage 
(cover multiple geologic units) 


¢ Post-recon plan ended up being 


= More flexible & adaptable 
= Recon data supports 
real-time replanning 
¢ Impact of recon 
= Reduced science uncertainty 
= Improved target prioritization 


T. Fong et al. (2010). Assessment of robotic 


recon for human exploration of the Moon. 
Acta Astronautica 67 (9-10) 
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¢ Never enough time for field work 
¢ “If only | could have...” 


= More observations 
= Additional sampling 


= Complementary & supplementary work 
The solution 


¢ Use robots to “follow-up” after humans 
¢ Augment human field work with 
subsequent robot activity 


¢ Use robots for work that is tedious 
or unproductive for humans to do 
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Follow-up 
Planning 


AT Dra 


Mission Planning » Crew Mission 


June 2009 July 2009 October 2009 

¢ Satellite images ¢ Two crews at ¢ Field data 

- ASTER, DEM, etc. Haughton Crater - Observations 
i and mission logs 


Human-robot teaming for space exploration 


» Robot Mission 


July 2010 
¢ K10 at Haughton 


¢ Science operations 


at NASA Ames 
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Haughton Crater 


¢ 20 km diameter impact structure 
¢ ~39 million years ago (Late Eocene) 


¢ Devon Island: 66,800 sq. km (largest uninhabited island on Earth) 
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Crew mission (July 2009) 
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Geologic Mapping Geophysical Survey 
¢ Document geologic history, ¢ Examine subsurface structure 
structural geometry & major units ¢ 3D distribution of buried ground 
¢ Example impact breccia & clasts ice in permafrost layer 
¢ Take photos & collect samples ¢ Ground-penetrating radar: 


manual deploy, 400/900 MHz 
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Stratified | ~~) 399] contact between 
sediments |) Me] carbonates 


View East ) Gray carbonate 
into crater eae >) breccia 
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Geophysical survey results 


distance (m) 
100 ‘ 180 


- subsurface ice wedges - 
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Field experiment results 


Explanation 

1) Alluvium 

Glacial Deposits 

(1) intracrater, Stratified Sediments 
() Grey Carbonate Breccia 


Geologic Mapping 


¢ Verified the geologic map in | 
multiple locations (revisited and Been A SO | svetescrnran 


1 Stratified Carbonate 1 


sth . Carbonate Marker Bed 


confirmed geologic units) 4 Fa Lo 
¢ Amended the geologic map in BR. (HOSA BEE; | mcr mec rn 
multiple locations (added detail 
to long-range crew observations) 


Geophysical Survey 


¢ Detail study of “polygons” 
(correlated surface & subsurface 
features identified by crew) 


¢ Measured average depth of 
subsurface ice layer (refined 
observations from crew) 


T. Fong et al. (2010). Robotic follow-up for 


human exploration. AIAA-2010-8605. 
Proc. of AIAA Space 2010. 
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Our focus 


¢ Study how humans can remotely support robots 


¢ Address the many anomalies, corner cases, and edge cases that 
require unique solutions, which are not currently practical to develop, 
test, and validate under real-world conditions 

¢ Humans provide high-level guidance (not low-level control) to assist 
when autonomy is inadequate, untrusted, etc. 


.... teaming for space exploration 


Future 


exploration architecture study teams have made assumptions 


about how crew can remotely perform work on a planetary surface ... 


ndidate Exploration Missions 

L2 Lunar Farside. Orion MPCV 

mission to Earth-Moon L2 point 
Near-Earth Asteroid. NEA dynamics 
and distance make it impossible to 
manually control robot from Earth 

Mars Orbit. Crew must operate surface 
robot from orbit when circumstances 
(contingency, etc.) preclude Earth control 


(NASA GSFC) 


Assumptions 


Productivity of crew-control (decision making, efficiency, etc.) 
Existing technology gaps (and how these can be bridged) 
Operational risks (proficiency, performance, failure modes) 
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Key Points 
¢ Demo crew-control of planetary 
rover from orbiting spacecraft 


¢ Test human-robot conops for 
future exploration mission 

¢ Obtain baseline engineering data 
(robot, crew, data comm, task, etc) 


SURVEY 


DEPLOY 


Implementation 
¢ Lunar libration mission simulation 
¢ Astronaut on Space Station 
¢ K10 rover in NASA Ames Roverscape 


INSPECT 


Expedition 36 testing 
¢ June 17, 2013 — C. Cassidy, survey 
¢ July 26, 2013 —L. Parmitano, deploy 
¢ Aug 20, 2013 — K. Nyberg, inspect 
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Orion MPCV at Earth-Moon L2 
¢ 60,000 km beyond lunar farside 
¢ Crew remotely operates robot 
¢ Does not require human-rated lander 


tea 


Lunar farside telescope 
¢ Lunar farside provides radio quiet 
zone for low-freq measurements 


¢ Requires surface survey, telescope Tso ein 3h 
ai a and inspection 


Earth 6, Ss 2: Trans-lunar Cruise, 5.5 Days 
10: Reentry, ~11 km/s 


9: Earth return cruise, 4.7 days Ll 


3: Lunar Flyby and Retro Burn “9: Lunar Flyby and 
——“K— Retro bu 
4: Cruise to L2, 4 Days 


: Loiter in Halo 
12 5 days 


8: Cruise to Moon, 
4.7 days 


Earth-Moon Rotating Reference Frame . 

Nominal Mission Duration: 31 Days 7: Halo Departure Burn 
Reference Launch Date: July 6, 2017 : 

Total Spacecraft AV after TLI: ~600 m/s 5: Halo Insertion Burn 
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Credit: Lockheed Martin / LUNAR 
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Why the EM-L2 Lagrange Point? 


EM-L2 is well situated for ee 
exploration of the Moon “a uN 
¢ Direct (line-of-sight) data / Fa \ 
communications to the lunar / # \ 
farside / Fs 
* Direct observation of lunar | L1 
L3 -- 
farside 1 + 0 
¢ AV can be lower than EM-L1 \ / Orbit 
- Demonstrate capability for \ ‘. / 
deep space operations in ‘ . / 
trans-lunar Space ss 
¢ Potential location for a “Deep 
Space Gateway’— staging 
point for future missions, cis- 
lunar science facility, etc. 
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ISS Expedition 36 


Surveying Telescope Telescope 
Deployment Inspection 


“—aPy 


Pre-Mission 
Planning 


Ps Ss ped Pie 
Crew gathers Crew monitors the Crew inspects and 
information needed rover as it deploys documents the 
to finalize the each arm of the deployed telescope 
telescope telescope array. for possible 
deployment plan. damage. 


1 
LA 


Ground teams 
plan out telescope 
deployment and 
initial rover 
traverses. 


Spring 2013 17 June 2013 26 July 2013 20 August 2013 


Chris Cassidy Karen Nyberg 
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Space Station test setup 


“Live” Rover 
ASY=Vakoxo) ar: Tale| 
Instrument 


Data 
(telemetry) 


Rover/ 
Science 
Data (e.g. 
imagery) 


fejsues 1 a|14 }$9}-1S0q ' 


Interface 
Instrumentation & 
Evaluation Data 


Rover Plan 


(command sequence) oe 2 i Se a : 
K10 rover at NASA Ames 
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Surface Telerobotics Workbench aan x 
Eile Help Alert Bar Rover Status 
Rower States 
Ak 2 O Newigoton §=© Hozerd Connecton Connected 
~~ | © Tesk Runner © Penoreene Commendebie Yes GPS 23Ape13 20:31:44 
Tip Prews skip to skip a task: press play to resume Task Sequence. Tip Bar O Fin Ons — 
Run Task Sequence Teleoperate Tab Panel Panorama 
Inspection 
Load Task Sequence ] “ 
Neme Tosti Bird’s Eye Top Down Restore 
Deserpton 3D View 3D View 
Primary 
EstOuraton 0002 30 Button 
Elopsed Time 00 00.09 
Sega ee Panel 


ceed 


| Duration | Comenand ae 


000123 Orne 


00.0001 Oeploye - - 
000010  Orve “fame ~¢ i Hozard Camera 


Run Task 
Sequence .. 


Controls 


Task 
Sequence 


Hazard 
Camera 


Status Bar 


23Ape13 21:31:36 sem command PAUSE _OVEVE 


Terrain hazards Rover camera 


& display 
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Surface Telerobotics Workbench — 
Ele Help 


Alert 11Apr13 00:26:00.875 Notice: Crewmember@SSC2 now controls robot 


Tip Send individual commands to the rover. 


Run Task Sequence Teleoperate 


& Teleoperate Controls 
Stop Rover j 
Forward 
50 cm j Im j 2m j 


Backward 
50 cm j Im 2m ] 


Rotate Left 


15° j 45° j go° j 


Rotate Right 


15° ; 45° ; 90° j 


Panorama Inspection 
Start ; Snapshot j 


| Teleoperate History 


Motion 
controls 


Camera 
controls 


| Time 


11Apr13 00:25:58 Ground override disengaged 


Human-robot teaming for space exploration 


Rover Status 
Ack 6 O Newigation © Hezerd Connection Connecte 
ow, © Tesk Runner © Penoreme Commendeble Yo: 
O Fillen O Inspection Battery 61.5% 


Controls Overlay 


Terrain hazards 


Rover path 


GPS 10Apri3 23:30:29 


| pee | 
teen | 


Restore 


Rover camera 
display 
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Chris Cassidy remotely operates K10 
from the ISS to perform site survey (2013-06-17) 
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K10 performing surface survey 
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Luca Parmitano works with K10 to deploy 
eS simulated polymide antenna (2013-07-26) 
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K10 deploying simulated polymide antenna 


i mnancovo teaming for space exploration 


Deployed simulated polymide antenna (three “arms”) 
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Karen Nyberg works with K10 to document 
Se deployed simulated antenna (2013-08-20) 
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K10 documenting simulated polymide antenna 


© Human-robot teaming for space exploration 


(Oi x=\\\exe) al ine) ie) im oem ROM ae)Y.>)0 


— a —— 


y : y i a, 
| / j : "| 
‘ y (un) | 
y j \ "pa = 

4 : y 1 q op eoetbetaaled 

: j ood _ 
- 7 v 

| 


SB somanrovo teaming for space exploration 


Assessment Approach 


Metrics 


¢ Mission Success: % task sequences: completed normally, ended abnormally 
or not attempted; % task sequences scheduled vs. unscheduled 


¢ Robot Utilization: % time robot spent on different types of tasks; comparison 
of actual to expected time on; did rover drive expected distance 


¢ Task Success: % task sequences per session and per task sequence: 
completed normally, ended abnormally or not attempted; % that ended 
abnormally vs. unscheduled task sequences 


¢ Contingencies: Mean Time To Intervene, Mean Time Between Interventions 
¢ Robot Performance: expected vs. actual execution time on tasks 


Data Collection 
¢ Data Communication: direction (up/down), message type, total volume, etc. 
¢ Robot Telemetry: position, orientation, power, health, instrument state, etc. 
¢ User Interfaces: mode changes, data input, access to reference data, etc. 
¢ Robot Operations: start, end, duration of planning, monitoring, and analysis 
¢ Crew Questionnaires: workload (Bedford Scale), situation awareness (SAGAT) 


automatic 
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Low-Latenc 


° Asteaneuis use Sboke as 
avatars to be remotely present 
at a field site 


¢ Focus on field science 
(emphasis on geology) 


Workshop #1: October 2016 
. Saviswed) state- of-the- art 
¢ Discussed pros and cons 


¢ Identified science goals 
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¢ Developed research roadmap 


¢ Identified key research and 
studies to be performed 


¢ Outlined summary report 
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KECK INSTITUTE FOR SPACE STUDIES 


http://kiss.caltech.edu/new_website/workshops/telepresence (_) ¢ 


f£SeERe 


Generously 
supported by 
the W. M. Keck 
Foundation 

# Home 
& Visit v 
1 Programs 


etre Space Science Opportunities Augmented by 


z Exploration Telepresence 


October 3 - 7, 2016 
California Institute of Technology - Pasadena, CA 91125 Wik 


Si 


Contact Us: 


Lectures 
Short Courses 
Symposia 
Special Events 
& People 


& Archive 
% (626) 
395-6630 
(626) 
395-3570 - fax 


i About 


Follow Us 


Email Us 


http://kiss.caltech.edu/new_website/ 
workshops/telepresence/telepresence.html 
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Many forms of human-robot teaming 
e “Robot as tool” is only one model 
¢ Not just co-located or line-of-sight 


> Humans & robots can support each other 
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¢ Human-robot interaction is still slow and 
mismatched (compared to human teams) 

¢ Easy for robots to impede the human 

>» Loosely-coupled teaming may be best 


Distributed teams 
¢ Require coordination and info exchange 


¢ Require understanding of (and planning for) 
each teammate’s capabilities 


> Effective protocols and tools are critical 


= Human-robot teaming for space exploration 
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Intelligent Robotics Group 
Intelligent Systems Division 
NASA Ames Research Center 


irg.arc.nasa.gov 
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